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Using Digital Mapping Tool in Ill-structured Problem Solving 


Abstract 

Scaffolding students’ problem solving and helping them to improve problem solving skills are critical in 
instructional design courses. This study investigated the effects of students’ uses of a digital mapping tool on 
their problem solving performance in a design case study. It was found that the students who used the digital 
mapping tool performed better than those who did not use the tool in providing arguments when defining the 
problem and justifying the solutions. The results indicated that digital concept mapping could facilitate 
students’ ill-structured problem solving. In this study guide map was used. Future research may examine the 
use of free style map in problem solving. 
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Abstract 

Scaffolding students' problem solving and helping them to improve problem solving skills are 
critical in instructional design courses. This study investigated the effects of students' uses 
of a digital mapping tool on their problem solving performance in a design case study. It 
was found that the students who used the digital mapping tool performed better than those 
who did not use the tool in providing arguments when defining the problem and justifying 
the solutions. The results indicated that digital concept mapping could facilitate students' ill- 
structured problem solving. In this study, guide map was used. Future research may 
examine the use of free style map in problem solving. 
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Introduction 


Ill-structured Problem Solving 

Instructional design is an ill-structured problem solving process (Jonassen, 2008). 
Ill-structured problems are characterized with unknown information, vague goals and open 
ending (Jonassen, 2005). Problem solving skills are not innate in learners, therefore, 
appropriate scaffolding needs to be provided for learners in problem solving process (Hung, 
2011). Ge and Land (2003) found that question prompts positively affected students' 
problem solving. Those who received question prompts significantly performed better than 
those who did not receive the prompts. The effectiveness of question prompts in supporting 
students' ill-structured problem solving was also reported in current literature (Bixler& 
Land, 2010; Chen, 2010). However, the limitations of question prompts were identified by 
some researchers (Chen & Bradshaw, 2007; Ge, Chen & Davis, 2005). The effective use of 
question prompts was influenced by students' prior knowledge or experience related to the 
problem, pre-existed assumptions about an issue, competence level in problem solving, and 
the amount of prompts. Question prompts may not function well for novice learners. Too 
many prompts could cause a heavy cognitive load. 

In addition to question prompts, the effectiveness of some other scaffolding 
strategies was discussed in current literature. These strategies include argumentation 
scaffolds in group problem solving (Cho & Jonassen, 2002), stories (Jonassen & Hernandez, 
2002), problem analysis guidelines (Ertmer et al., 2009), and relevant cases (Bennett, 
2010 ). 


Ertmer et al. (2009) explained that instructional design problems "typically require 
designers to understand, simultaneously, the specifics of a given situation (e.g., context, 
learners, tasks) as well as the multiple components of the larger system in which they are 
embedded (e.g., project constraints, relevant technologies, multifaceted learning goals)" (p. 
117). In their conceptual model of instructional design expertise, Stepich and Ertmer (2010) 
specified that problem finding and problem solving were two major components in expert 
thinking. Compared to novices, experts demonstrate better understanding of the nature of 
the problem and are more able to articulate problem situation in coherent way by 
integrating their prior knowledge with new information. It was suggested that novices 
construct mental representations of elements and issues in problem situation in order to 
solve a problem (Jonassen, 2005). 

Concept Mapping 

Unguided exploration in a highly complex environment may cause cognitive overload, 
which in turn, will negatively affect learning (Kirschner, Swell & Clark, 2006). Given the 
complexity of ill-structured problems, it is necessary to scaffold learners' cognitive 
processing in their problem solving. Jonassen (2005) suggested problem be organized and 
displayed to students "in ways that enhance their mental representations and engage 
appropriate problem-solving process" (p. 85), such as using concept maps. 

According to Canas et al. (2005), "Concept maps are a graphical two-dimensional 
display of knowledge that is comprised of concepts (usually represented within boxes or 
circles), connected by directed arcs encoding brief relationships (linking phrases) between 
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pairs of concepts. These relationships usually consist of verbs, forming propositions or 
phrases for each pair of concepts" (p. 206). The visual format of concept maps externalize 
learners'conceptual knowledge, represent information in structure, and therefore facilitate 
learners' mental activity. Wilgis and McConnell (2008) found that concept mapping was an 
effective strategy improving graduate nurses' critical thinking and decision-making skills. 
They concluded that this strategy helped novices to perform like competent professionals. 
Mapping strategy was also reported to be useful in promoting argumentation skills and 
resulted in more refutations of opposing arguments (Nussbaum & Schraw, 2007). 

With the development of computer technology, digital concept mapping tools make 
knowledge and information visualization more accessible and interactive. Compared to 
traditional concept maps, digital concept mapping tools allow free editing when 
"(re)constructing, (re)organizing, and (re)representing mapped knowledge" (Tergan, 2005, 
p. 194), as well as direct access to source of information through the use of hyperlink. The 
enhanced usability functions make digital concept maps more suitable for representation, 
visualization and construction of knowledge. It was reported that digital concept maps 
facilitate collaboration in problem solving by external representation of all collaborators' 
knowledge and information (Engelmann & Hesse, 2010). 

Jonassen (2005) concluded that "It is necessary to understand the conceptual 
relationships between the concepts in any problem domain in order to be able to transfer 
any problem-solving skills developed" (p. 88), therefore, the effects of digital concept maps 
on problem solving need to be examined. 


Purpose 

The purpose of this study was to investigate how using a digital concept mapping tool 
could affect students' problem solving in instructional design. Specifically, two hypotheses 
were examined: 

1. The students who used the digital mapping tool performed better in problem solving 
than those who do not use it. 

2. The students' mapping performance was positively related to their problem solving 
performance. 


Methods 


Participants and Setting 

The participants in this study were 26 undergraduate students who enrolled in an 
instructional technology course in a four-year college. There were two class sections. 
Section one had 16 students and section two had 17 students. Each section met once a 
week throughout the semester. Before taking this course, the students had taken an 
instructional design course. All the students in section one and 10 students in section two 
participated in this study. The students in section one were randomly assigned to 
experimental group, and the 10 students in section two were in control group. 

The purpose of the present study was to examine the effect of digital mapping on 
students' problem solving. The mapping tool used in this study was Bubbl.us, an online 
mapping tool. In Bubbl.us, the map begins with a color bubble icon that allows users to put 
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in main topic. Using this tool, the users create maps in which relevant concepts and ideas in 
bubble icons can be linked with arrows and phrases to show specific relationships. The 
bubbles at different levels enable hierarchical structure or web structure of the map. All of 
these characteristics are typical of concept maps (Davies, 2011). The map constructed 
online can be converted to image format. 

Data Collection 

The students in both groups were given an instructional design case to read, analyze 
and asked to write a report. In the report, they were required to analyze the problem and 
situation, proposed solutions, and made justifications. 

Before working on the instructional design case, the treatment group learned to use 
Bubbl.us to design concept maps. Since concept mapping is "a cognitively demanding 
activity, and that the process of map modification may sometimes be messy and 
cumbersome" (Tergan, 2005, p. 199), the instructor constructed a map template (Figure 1) 
and showed it to the group. Using this map as reference, the students created concept 
maps using Bubbl.us. Specifically, they created pre-defined bubbles and then filled in the 
bubbles with their claims and evidences. 


When completing the activity, the students in the treatment group submitted reports 
and maps. Without using the mapping tool, the students in the control group only submitted 
reports at the end of the activity. 



Arguemnts 


j 


Arguments 


J 


Arguments 


J 


Figure 1. Map template 


Data Analysis 


A scoring system was developed to measure the students' problem solving 
performance. The system contains four aspects of problem solving: 1) define the problem 
situation, 2) provide arguments when defining the problem, 3) develop solutions, and 4) 
justify solutions by providing arguments. In each group, the students' reports were 
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assessed using this scoring system. Besides the score on problem solving, average score in 
each aspect was calculated. 

The map template provided by the instructor was used as criteria to evaluate and 
grade the concept maps constructed by the students in the treatment group. Specifically, 
the grading was based on 1) whether the map showed the major issues in the problem and 
supportive arguments, 2) whether the map showed the solutions and supportive arguments, 
and 3) relationships between concepts by using arrowed lines and linking word(s). 

To test the first hypothesis, two sample t-tests were conducted to see whether the 
treatment group had significantly higher scores than the control group. Meanwhile, the 
students' scores in each aspect of problem solving were also compared between the two 
groups. Before each t-test, F-test was conducted to determine whether the two groups have 
homogeneous variances. To test the second hypothesis, linear regression analysis was used 
to examine whether mapping scores were the predictor of the students' scores of the 
reports. 


Results and Discussions 

The students' mean scores in each aspect of problem solving were calculated. Table 
1 shows the results of each group. 

Table 1 

Students' Mean Scores and Standard Deviations 


Groups 


Define the 

problem 

situation 

Provide 

arguments 

when 

defining the 
problem 

Develop 

solutions 

Justify 

solutions 

Overall 

problem 

solving 

performance 

Experiment 

M 

1.25 

1.81 

1.63 

1.25 

5.94 

N - 16 

SD 

0.45 

0.91 

0.5 

0.77 

2.05 

Control 

M 

0.8 

1 

1.4 

0.8 

3.9 

N - 10 

SD 

0.79 

1.05 

0.52 

0.82 

2.64 


It can be seen that the students in the experiment group earned higher scores than those in 
the control group in each aspect of problem solving and overall performance. 

Hypothesis One: The students who used the mapping tool performed better in 
problem solving than those who did not use it. 

To test this hypothesis, F-tests were first conducted. The results indicated unequal 
variance of the two groups in defining the problem situation (p = 0.025), but equal 
variances in providing arguments (p = 0.296), developing solutions (p = 0.438), justifying 
solutions (p = 0.4), and overall problem solving performance (p = 0.18). Accordingly, t- 
tests were performed to see whether there was any significant difference in the students' 
performance. The results showed that the students in the experiment group and the 
students in the control group demonstrated significant difference in their problem solving, 
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£[24] = 2.021, p = 0.02, as expected, the experiment group significantly performed better 
than the control group. When examining each aspect of problem solving individually, 
significant differences was found in providing arguments when defining the problem 
situation (£[24] = 2.08, p = 0.02), and justifying solutions (£[24] = 1.72, p = 0.049). No 
significant difference was found in defining the problem and providing solutions. This means 
the experiment group demonstrated better performance in providing arguments than the 
control group. 

Jonassen and Kim (2010) argued that helping learners to visualize arguments could 
facilitate their argumentation which in turn would improve their problem solving. In this 
study, the students in the experiment group used Bubbl.us to construct concept maps that 
presented major issues of the problem, arguments to support the identification of the 
issues, solutions developed, and arguments to justify the solutions. The results indicated the 
effectiveness of visualization of arguments on problem solving. A close examination of the 
students' reports revealed that 4 out of 10 students in the control group did not provide 
arguments when defining the problem situation, and 5 students did not argue for their 
proposed solutions. On the contrary, only 2 out of 16 students in the experiment group did 
not provide arguments when defining the problem situation, and 2 students did not justify 
their solutions. 

The students in the two groups were given the same directions when working on the 
instructional design case, except that the experiment group were required to construct 
concept maps in problem solving process. The concept map template provided by the 
instructor specified the components including major issues, solutions, arguments and 
relationships ("supported by"). The students used it to gather relative information and 
checked that all necessary information is identified according to the pre-defined 
relationships. This could have scaffolded the students' cognitive and metacognitive 
processing of information and knowledge in problem solving. 

Hypothesis Two: The students' mapping performance was positively related to 
their problem solving performance. 

To test the second hypothesis about the relationship between the students' mapping 
performance and their problem solving performance, single regression analysis was 
conducted, with the mapping scores being independent variable and the overall problem 
solving performance score being dependent variable. The analysis did not yield significant 
relationship between the two variables (p = 0.1). This indicated that the students' mapping 
performance did not explain their problem solving in this study. Possible reason for the lack 
of relationship is that the students used mapping template to create concept maps, instead 
of producing maps of their own style. 

Dansereau (2005) defined three general types of maps: Information, guide, and free 
style. Information maps are complete maps constructed by an expert. Guide maps are "fill- 
in-the-blank" (p. 62) tools in which the layout of the map, the nodes with labels, and the 
links are created by an expert. What the users need to do is to fill out the nodes. Free style 
maps are the maps that are created by the users themselves. In this study, the map 
template provided by the instructor served as a guide map in which the structure of the 
map, the linked and labeled bubbles are all presented to the students. The students created 
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maps of the same structure with bubbles and then inserted the text in the bubbles to 
complete the map. 

Due to the fact that all the students in the experiment group used the same guide 
map as a template, most of the students' concept maps followed the same format, although 
some students did not show all the required information in their maps. The evaluation of the 
students' mapping performance was essentially on the completion of the guide map. 
Therefore, the evaluation was more about the format of the map. In such situation, it is 
understandable that the students' performance in mapping did not predict their problem 
solving performance. 


Implications and Recommendation 

There is clear relationship between argumentation and problem solving (Jonassen & 
Kim, 2010). Therefore, scaffolding students'argumentation is essential in problem solving. 
The findings in this study implied that using digital concept mapping tool could facilitate 
students' argumentation. Those who used the tool performed significantly better than those 
who did not use the tool when providing arguments to define the problem and justifying 
solutions. Davies (2009) stated that "we become especially cognitively overloaded when 
dealing with arguments. Arguments are constructed in words and sentences and 
(importantly) the inferential links being made between the sentences" (p. 803). Using a 
digital concept mapping tool facilitates construction of arguments by making writing and 
editing easier. Meanwhile, organizing the components in the map allows students to self¬ 
explain, externalize and visualize their understanding of information when constructing 
arguments. 

In this study, the students in the two groups did not differ in defining the problem 
situation and providing solutions. Choi and Lee (2009) summarized that epistemological 
beliefs, metacognition, argumentation skills, and domain knowledge were essential skills 
when solving ill-structured problems. All the participants in this study took an instructional 
design course before taking this technology course. They had domain knowledge in 
instructional design. Digital mapping facilitated the students' argumentation in experiment 
group. However, the students' epistemological beliefs and metacognition were not in the 
scope of this study. Future research may examine these two variables in problem solving to 
find out the effects of these skills on students' problem solving when defining the problem 
and proposing solutions, in relation to the use of concept mapping strategy. 

Self-constructing concept maps highly demands cognitive efforts in analyzing and 
externally representing knowledge and information (Tergan, 2005). Therefore, guide map 
was used in this study. However, this made it hard to determine whether the students' 
mapping performance could predict their problem solving. Conceicao and Taylor (2007) 
found that personal learning preference, difficulty of concepts, and design of the concept 
map influenced creation of concept maps. Research is needed to examine students' self¬ 
construction of free style concept maps in instructional design problem solving activity. 
Hopefully, such research would help to answer the question about the relationship between 
students'mapping performance and problem solving performance. 
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Conclusion 

External representation of problems can reduce students' cognitive load when solving 
ill-structured problems (Jonassen, 2005). Concept maps are effective in representation and 
visualization of information and ideas. Computer-based digital concept maps have enhanced 
usability than traditional maps due to digital features. Tergan (2005) summarized that 
"Analysis of the potential of digital concept maps as tools for managing knowledge and 
information, as well as suggestions for their effective use in resource-based learning and 
problem-solving scenarios, are long overdue" (p. 200). In current literature, although it was 
suggested that graphic tool could scaffold argumentation (Jonassen & Kim, 2010), few 
studies specifically investigated the effects of digital concept maps on ill-structured problem 
solving performance in instructional design. This study intended to contribute to the field by 
providing researchers and practitioners with empirical data. It is in the hope that more 
research will be conducted to further explore this issue and find out how to better use digital 
concept mapping tools to facilitate instructional design problem solving. 
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